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ABSTRACT 



This paper suggests that the variety of decision 
making proposed by Professor Lortie will not afford the luxury of 
evaluative systems of the kind he describes. Professor Gordon feels 
that, had Professor Lortie pursued a line of functional analysis of 
many outcomes, he would have arrived at an entirely new analysis of 
the justification for complexity of evaluation., (Author) 
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The CENTER FOR THE STUDY OF EVALUATION OF INSTRUCTIONAL 
PROGRAMS is engaged in research that will yield new ideas 
and new tools capable of analyzing and evaluating instruc- 
tion. Staff members are creating new ways to evaluate con- 
tent of curricula, methods of teaching and the multiple 
effects of both on students. The CENTER is unique because 
of its access to Southern California's elementary, second- 
ary and higher schools of diverse socio-economic levels 
and cultural backgrounds. Three major aspects of the pro- 
gram are 

Instructional Variables - Research in this area 
will be concerned with identifying and evaluating 
the effects of instructional variables, and with 
the development of conceptual models, learning 
theory and theory of instruction. The research 
involves the experimental study of the effects of 
differences in instruction as they may interact 
with individual differences among students. 

Contextual Varjahles - Research in this, area will 
be concerned with measuring and evaluating differ- 
ences in community and school environments and the 
interactions of both with instructional programs. 

It will al,so involve evaluating variations in stu- 
dent and teacher characteristics and administrative 
organization. 

Criterion Measures - Research in this field is con- 
cerned with creating a new conceptualization of eva- 
luation of instruction and in developing new instru- 
ments to evaluate knowledge acquired in school by 
measuring observable changes in cognitive, affective 
and physiological behavior. It will also involve 
evaluating the cost-effectiveness of instructional 
programs . 
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COMMENTS ON PROFESSOR LORTIE'S PAPER ENTITLED 
"THE CRACKED CAKE OF EDUCATIONAL CUSTOM 
AND EMERGING ISSUES IN EVALUATION" 

C. Wayne Gordon 

Dr. Lortie's shadow has preceded him; his influence is already 
apparent in its effect on evaluation. From what I have heard around 
the country he has had his finger in the cake and has been helping 
practice the practice of evaluation in his own right. 

He disturbs me, however; and this is not a prologue to saying 
I agree with everything he said, with the monstrous complexity with 
which he confronts us . 

At this point, the evaluator must feel like the centipede who 
looked out in the morning to see which foot to put down and had to 
make a decision. He was so cantonized he didn't get up all day. 

Fortunately, Lortie has given us a foot or two to put down. 

I would like to put one or two of them down at the risk of recapping 
the main set piece for discussion- -that is, change. He rather 
belatedly introduced the dynamic force for economic change in the 
new discussions and the new discoveries, but the change itself has 
taken on a value. It is like the new tide- -better than the old. 

He gives us a set of competing sources for the alternatives 
we seek. These are largely external to the system- -government 
sources, commercial activism, and even the universities with their 
wares . 

These competing sources are not just standbys; they are sources 
of control as well. These are not alternatives which the educator- 
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administrator is free to accept without some coercion. He is 
obliged to get a grant to develop his particular program. To that 
extent we do have a new situation, I think; and one which has powerful 
control implications, powerful for elevating the role of evaluation. 

So, we have this harassed educator- administrator who must make 
decisions; and because there is contention in the system he must 
make not only correct decisions, but ones that have legitimacy. 

These won’t necessarily be the same. In other words, the most accept- 
able answer and the basis for having supplied it will not always 
depend on the validity of the answer or the evidence used to support 
it. 

The professional evaluator is going to be the handmaiden who 
plays the appropriate pattycake with the educator-administrator’s 
problems. He is like an umpire in the final game of the World 
Series. He not only has to make the right decision, but he has to 
have good eyesight. 

So, we come to integrity; we must have integrity in this process. 
But because the administrator has to make choices, we are in the 
short run faced with heavy demands on evaluation services . Their 
decisions have to be defended in the face of competing alternatives. 

We have set the stage for objectivity and rationality in science 
and truth. For the first time we have a structural determinism 
which forces people to be both honest and correct because of the 
disadvantages of bad choices and bad decisions. We have powerful 
resources which place demands on us at the highest level of inte- 
grity- - science and truth. 

I notice that Lortie doesn’t believe that he wants that much 
truth or that much knowledge. He says, ’’The thoughtful school 
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board members, administrators and teachers will be skeptical of 
plans made by sponsors of any new approach”; and, ’’The less thoughtful 
colleagues may find that the public expects them to appear as if they 
are giving careful consideration to new policy builders.” Here I 
find the latter point of view the more typical possibility. 

I just don’t think this variety of decision-making will afford 
the luxury of evaluative systems of the kind he describes. The 
demand in this market isn’t entirely for the kinds of product he 
suggests . 

Now, even so, it seems to me that he has opted for a classic 
solution. It’s for rationality; it’s bureaucratic; he has decisions 
to be made based on expertise, honesty, and integrity. And we have 
a profession prepared to provide this expertise. But then he in- 
troduced a new problem. He said, ”He not only needs answers, but 
he needs answers to questions which are infinitely more complex 
than the ones which he has decided.” Lortie hinted at the problem 
of functional analysis of ^ any outcomes, including unintended effects 
of instruction. If he had pursued this line of reasoning, he would 
have been led into an entirely new analysis of the justification for 
complexity of evaluation. I then could have accepted his concern 
for expanding the range of variables and values. 

That is where my centipede arises. The user, the educator- 
administrator, doesn’t want that much knowledge. Yet we must, as 
Professor Gage says, talk about basic research methods rather than 
value judgments. The administrator needs quicker, shorter, and 
more simple-minded answers. 
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I have another concern. It is for the evaluator because 
Lortie has made his problem complicated. The evaluator has a stock 
in trade. In most cases he has a gimmick, a system, a line of goods 
that he is prepared to provide in this market. We have some symmetry 
in the insincerity of the request for evaluation and the incapacity 
of the evaluator to fulfill the demand. 

I think the evaluator is a bit like the policeman who found a 
dead horse on Potawotomi Street. He couldn't spell "Potawotomi" 
and had to make a report; so he dragged the horse to First Street 
and wrote the report. 

If the demand for the rationality, if the integrity of the 
answer is not sought to that extent, the educator-administrator can 
probably do well at a different kind of hot dog stand. He will 
seek someone more willing to serve in this market, who will not 
impose on hini the complexity which Sortie suggests, nor will he impose 
on him the embarrassment of going to the public with his problem. 

Lortie has added another complexity which is even more difficult, 
and that is elaborate studies designed to fit the uniquely different 
facts of the case. This again is messy, difficult, demanding, time 
consuming, and costly. As you try to prescribe for him the package 
that will be important for his immediate need, the evaluator is not 
listening. He is waiting for another kind of advice- -how to get the 
contract renewed. 

Now, may I give one more thought? Lortie suggests a rather 
facile way to get out of the dilemma of how the evaluator can avoid 
being caught providing unpopular conclusions because of the values 
present in the package. Lortie has suggested a weighted system of 
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assessing the results in terms of different value perspectives. He 
neutralizes the necessity for having values themselves. The eval- 
uator is going to be an operator of relative perspectives from which 
he derives objectives to assess. 

Maybe this is the only way to avoid the dilemma. But would you 
be willing to consider the kind of optimism which says, let’s include 
enough of the dependent variables in the value area- -a smorgas- 
bord of values- -i . e. , let's get some criteria in there that we 
haven't been asking about; and then let's observe which independent 

variables are probable for the best predictions. Could we expect 

eventually to get a sufficiently generalized system of variables on 
both sides of the systam, independent and dependent, that would 
achieve an objectivity in which even the evaluator or the researcher 
would propose values and possible effects that transcend the parochial 
foci of controversy?. At least then we could rest a bit on the 

possibility that we are not so subject to the winds of the conflict 

and the controversy that he proposed that we have nothing to offer 
but to follow the fads that are going to be proposed. 




